1. Field of the Invention
The present invention relates to distributed storage of data in a secure and fault-tolerant manner and retrieval of such data from distributed data stores using fault-tolerant recovery techniques.
2. Description of the Related Art
Data storage and retrieval technology requires the availability of data in a timely manner. Basic data storage techniques involve generating a copy of the original data as a backup: such backup systems include simultaneous copying to two or more storage locations (e.g., simultaneous copying to two hard drives), and archival of data. Data archival has progressed from tape backup systems, to backups using compact disc (CD-R) technology, etc.
Such storage and retrieval techniques are substantially inefficient in terms of processing requirements, disk space, and time constraints. For example, distributed storage systems typically maintain duplicate copies of data on multiple, disparate machines to avoid failures in the case of one or more nodes fails. The distribution of duplicate copies, also referred to as r-replication, copies the data, in whole, among R separate storage devices in a system. In case of a failure, any one of the nodes may service a request for data.
The use of r-replication may be effective for closed storage services, such as servers having a Redundant Array of Inexpensive Disks (RAID), also referred to as RAID servers, or corporate mirroring servers. However, r-replication cannot be implemented efficiently in ad hoc or unreliable networks such as the Internet because each replication substantially increases the total storage requirements of the data; hence, typical implementations of r-replication use a minimal number of copies (e.g., a RAID 0 system uses only two copies (R=2).
In particular, use of an r-replication system is extremely inefficient if a given storage device is available on average only fifty percent of the time: if two storage nodes have a fifty percent availability, then the aggregate guaranteed data availability is limited to seventy-five percent for two copies (R=2). In other words, in order to guarantee ninety-five (95) percent availability, five copies (R=5) of the data would be required, effectively limiting the storage capacity of a system to twenty percent its total capacity. Further, the necessity of multiple read requests ordered sequentially to the duplicate storage devices substantially reduces the throughput of the system, especially each time a read request fails.
Another problem in using r-replication involves rogue nodes that maliciously or inadvertently return incorrect data to a requesting source (e.g., due to read/write errors or transmit/receive errors). Security against rogue nodes requires additional redundancy within the system, requiring an even higher number of duplicate copies to be added to the system.
Other problems associated with data storage involve large scale recovery of data, for example due to a disaster recovery scenario. Typical systems that rely on a centralized data store run the risk of complete data loss in the event the data storage is damaged or destroyed. Hence, conventional redundancy-based replication systems may be ineffective in the case that all the data stores are located within a specific geographic area having encountered a disaster (e.g., fire, etc.).
Still another problem associated with data storage involves the relative portability of data and the ability of users to access the data from different locations. One example involves an enterprise system having multiple offices, where a user moves from one office to another office. Conventional systems require a complete reconfiguration of the user's portable computer before access to any data store (e.g., e-mail) is possible.